SHAP Features Tool Overview¶
This notebook outlines basic use of the 'SHAP' Python package to analyse Machine Model models, in terms of determinign feature signifance.
Model interpretation is the key value gained from this tool.
What are SHAP Features?¶
Model Interpretation
Historically, interpretability of feature importance has been straightforward with linear models. Often the goal is to understand the parametric value associated with a given feature. The higher the value, the greater its impact on the overall model at a global level. This is useful, but is limited in value across the tapestry of Machine Learning techniques. SHAP is a tool / approach that intends to aid in this generic interpretation.
SHAP - Game Theory Basis
This technique is based on game theory, lets start with an overview of the key beats of this in particular:
A group of people are playing a game. As a result of playing this game, they receive a certain reward; how can they divide this reward between themselves in a way which reflects each of their contributions? They would follow rules like so:
- The sum of what everyone receives should equal the total reward
- If two people contributed the same value, then they should receive the same amount from the reward
- Someone who contributed no value should receive nothing
- If the group plays two games, then an individual’s reward from both games should equal their reward from their first game plus their reward from the second game
In a machine learning problem, the reward is the final prediction of the complex model, and the participants in the game are features.
The Shapley Equation
At a very high level, what this equation does is calculate what the prediction of the model would be without feature i, calculate the prediction of the model with feature i, and then calculate the difference.
It does this by constructing my sets S. These are all possible combinations of features, from which we can derive the importance of a given feature in N dimension feature input spaces.
You can imagine the granularity in difference will have high variance in models such as nueral networks, due to what is often a numeric output.
Note: You can also imagine in certain algorithms that the feature is encountered first may have the highest weighting / bias. For example as the splitting attribute in a decesion tree. The construction of sets of sets resolves this bias.
The SHAP library introduces optimisations to make the calculation of SHAP values more effecient, so the brute force approach does not exactly occur in practice but is good to reason with.
Depedencies¶
! pip install shap
! pip install xgboost
! pip install pandas
! pip install sklearn
! pip install tensorflow as tf
! pip install xgboost
from xgboost import XGBClassifier
import sklearn
import pandas as pd
import shap
import tensorflow as tf
from tensorflow.keras.preprocessing import text
from sklearn.preprocessing import LabelEncoder
Data Acquisition & Preparation¶
The dataset is stock, irrelevant in this columnar experiment.
url="https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data"
df = pd.read_csv(url)
# Add Column Names
df.columns = ['sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species']
df.sample(5)
Create XGB Model¶
labels = df['species']
data = df[['sepal_length','sepal_width', 'petal_length', 'petal_width']]
train_data, test_data, train_labels, test_labels = sklearn.model_selection.train_test_split(data, labels, random_state=2)
model = XGBClassifier()
model.fit(train_data, train_labels)
Create SHAP Tree Explainer & Demonstrate Plotting¶
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(train_data)
In the next code cell below, you can see that the SHAP explanation can be acquired on a class by class basis, through selecting the appropriate index.
The data structure of shap_values is such that each class is a list, within each of those lists is a list of SHAP values for each training data instance. Representing the core coeffecients for each feature that impacted the classification during training.
Explanation
The 'output value' is the prediction for that observation.
The 'Base Value' is the value that would be predicted if we did not know any features for the current output. Its the mean model output.
Red/blue: Features that push the prediction higher (to the right) are shown in red, and those pushing the prediction lower are in blue. These represent 'forces'
The larger the segment associated with a feature, the greater impact it has had on the models output value.
class_selection_by_index = 1
instance_index_selection = 25
shap.initjs()
shap.force_plot(explainer.expected_value[class_selection_by_index], shap_values[class_selection_by_index][instance_index_selection],
train_data.iloc[instance_index_selection])
Explanation
The top X axis is each data point in the training set.
The three classes are distributed across the range of the model output value, with class label in specific bins of that range.
Each different class has a different degree of feature force, in combination. Your can see that 'petal_length' is a key discrimintory feature between at least two of the classes.
In the X value of '50' you can see 'sepal_length' tries to push the feature combination into a higher model output value, but the strength of the other features ensures a consensus classification.
You can think of each 'Y' as a individual force plot as seen in the prior example.
shap.initjs()
shap.force_plot(explainer.expected_value[class_selection_by_index], shap_values[class_selection_by_index], train_data)
Explanation
Shap auto selects another feature to show interaction with our target feature with another, in this example the 'sepal_length' was selected by SHAP.
The two feature values for each data point are plotted.
This plot shows that for petal length there are three clear groupings of shap values.
A first grouping of petal length has a negative shap value (blue) which mean this is opposing the target class in the first grouping.
The second grouping of moderate petal lengths show that this length typical has a mixed impact on class assignment, showing that it could be hindering the model as it influences the target class positively and negatively depending on the instance.
The final grouping tells us that the higher petal length almost always has a positive contribution to a target class (seen by mainly red's).
feature_to_plot = 'petal_length'
shap.initjs()
shap.dependence_plot(feature_to_plot, shap_values[class_selection_by_index], train_data)
Explanation
The plot shows which features are important for each class.
In the case of 'sepal_width' we can see that it only impacts class two assignment in any notable way.
While 'petal_length' has a key impact on all of the class assignments.
Given its higher shap value, it is in general the most important variable in the dataset.
shap.initjs()
shap.summary_plot(shap_values, train_data)
Explanation
To get an overview of which features are most important for a model we can plot the SHAP values of every feature for every sample. The plot below sorts features by the sum of SHAP value magnitudes over all samples, and uses SHAP values to show the distribution of the impacts each feature has on the model output.
This plot shows the most important feature for this class in order.
The Y points represent every SHAP value associated with a prediciton, showing if it has a role in a positive or negative contribution to the target variable.
We can tell sepal_width is not discriminatory for this class.
For each point associated with the 'petal_length' we can see that it has a high degree of important at its value poles. Indicating its a key discrimintory variable for this class.
If you change the index to '0' you will see that 'petal_length' is the only feature which is key for determining the class.
shap.initjs()
shap.summary_plot(shap_values[class_selection_by_index], train_data)
shap.initjs()
shap.summary_plot(shap_values[0], train_data)
SHAP Kernel Explainer (Any Model Type)¶
Kernel SHAP uses a specially-weighted local linear regression to estimate SHAP values for any model
It uses test data to get SHAP values, this differs from the training of a tree.
svm = sklearn.svm.SVC(kernel='rbf', probability=True)
svm.fit(train_data, train_labels)
explainer = shap.KernelExplainer(svm.predict_proba, train_data, link="logit")
shap_values = explainer.shap_values(test_data)
# plot the SHAP values for the output of the indexed instance
shap.initjs()
shap.force_plot(explainer.expected_value[class_selection_by_index], shap_values[class_selection_by_index][instance_index_selection,:], test_data.iloc[instance_index_selection,:], link="logit")
NLP SHAP'ing¶
Text Data Load¶
uri="sample_data/spam.csv"
df = pd.read_csv(uri, encoding = "ISO-8859-1")
# Drop dangling columns and rename remainer to human readable terms.
df = df[["v1", "v2"]]
df.columns = ["label", "text"]
df.sample(5)
Prepare Data for Modelling¶
VOCAB_SIZE = 400
NUM_OUTPUTS = 2
Label encode our targets.
LE = LabelEncoder()
df['code'] = LE.fit_transform(df['label'])
df.sample(5)
Split our data into training and testing sets
data = df['text']
labels = df['code']
train_data, test_data, train_labels, test_labels = sklearn.model_selection.train_test_split(data, labels)
Tokenise and generate BOW - basic one hot encoding array.
In a comment I have a tokenizer with no vocabulary limit, this will need changed in tandem with the size of the NN's first layer later if you want to use all the words. For more info, see limitations.
tokenizer = text.Tokenizer(num_words=VOCAB_SIZE)
# tokenizer = text.Tokenizer()
tokenizer.fit_on_texts(train_data)
bag_of_words_train = tokenizer.texts_to_matrix(train_data)
bag_of_words_test = tokenizer.texts_to_matrix(test_data)
Modelling¶
Define the architecture of a basic neural network to accept len(BOW) and output num_classes.
Use sparse crossentropy loss function when there are two or more label classes. Expect labels to be provided as integers.
# bag_of_words_vector_len = len(bag_of_words_train[0])
# Define Neural Network.
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(50, input_shape=(VOCAB_SIZE,), activation='relu'))
# model.add(tf.keras.layers.Dense(50, input_shape=(bag_of_words_vector_len,), activation='relu'))
model.add(tf.keras.layers.Dense(25, activation='relu'))
model.add(tf.keras.layers.Dense(NUM_OUTPUTS, activation='sigmoid'))
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
What Does our Model Look Like?
model.summary()
Train our model...
model.fit(bag_of_words_train, train_labels, epochs=3, batch_size=32, validation_split=0.1)
Is our Model any good?
model.evaluate(bag_of_words_test, test_labels, batch_size=32)
SHAP Analysis¶
Setup our SHAP explainer object.
explainer = shap.DeepExplainer(model, bag_of_words_train)
Then we’ll get the attribution values for N individual predictions.
NUM_PREDICTIONS = 10
shap_vals = explainer.shap_values(bag_of_words_test[:NUM_PREDICTIONS])
Get Words in Vocabulary¶
Get all the words in the dataset, for reference later.
words = tokenizer.word_index
word_lookup = list()
for i in words.keys():
word_lookup.append(i)
word_lookup = [''] + word_lookup
What features have the greatest impact on the classification?
We can see 'is' and 'you' are indidactors of HAM while 'year' and 'tonight' are SPAM orientated, however due to the data imbalance, this is only part of the picture.
shap.initjs()
shap.summary_plot(shap_vals, feature_names=word_lookup, class_names=LE.classes_)
For a given class, discover the impact of the features on the model output for each time the word occurs for this class.
class_selection_by_index = 1
shap.initjs()
shap.summary_plot(shap_vals[class_selection_by_index], feature_names=word_lookup, class_names=LE.classes_)
Show the SHAP Values for a single predicition. See limitations on why the labels are binary values, rather than text.
We can see here a sequence of features drive the model value output down, influencing the class assignment.
instance_index_selection = 2
shap.initjs()
shap.force_plot(float(explainer.expected_value[class_selection_by_index]),
shap_vals[class_selection_by_index][instance_index_selection],
bag_of_words_test[instance_index_selection])
Limitations¶
- The combinatory use of the Keras Tokenizer and SHAP makes it difficult to use a limited volcabulary in order to visualise the originals words represented within the BOW data structure. There is no clear conversion from BOW to text items in the Tokensier, which leads to showing in SHAP plots the BOW one hot values.
- If you remove the restriction of a limited vocabulary this would lead to plotting a graph of with too many possible variables, SHAP does not render this correctly.
- Ordering the word counts from the Tokeniser is potentially feasible but its is not trivial to determine if the order of each instance in the BOW corresponds to a word frequency order of the text to enable this.
- To this end, I have accepted the limitations and showed the results as is.
- It is possible to use other tooling in future (Not Keras, at least this verion of it) to enable easier retrieval of the original words from a BOW vector